Click any tag below to further narrow down your results
Links
OpenAI’s data productivity team built Kepler, an AI agent that understands their data platform context and writes iterative SQL to answer complex data questions. It handles table discovery, query refinement, follow-up prompts and even visualizations, cutting analysis time and reducing errors.
- OpenAI built Kepler, an internal AI agent that writes iterative SQL to answer data questions across 70,000 datasets and 600 petabytes daily, deployed via Slack, IDEs, or web UI
- It doesn't just query once—it discovers tables, refines CTEs, verifies numbers against dashboards/Notion, and proposes hypotheses (e.g., duplicate logging) for anomalies like a WAU spike
- Memory for self-learning plus rigorous evals let it improve over time without regressing
- Rolled out to 80% of the company, turning multi-engineer, multi-hour investigations into minutes
Hex built a suite of analytical evals to test data-analysis models and found Claude Fable 5 outperforms its Opus 4.x predecessors by 10–15%, nailing both semantically modeled and raw-data tasks with fewer mistakes. They’ve also designed a tougher “Frontier” benchmark for long-horizon, open-ended scenarios, where Fable 5’s careful assumptions and cross-checks boost its pass rate to around 58%.
- Claude Fable 5 beats Opus 4.7 by 10-15 points on Hex's core benchmarks, scoring 93%+ on Analytical Hard/Semantically Modeled tests and 65% on Semantically Unmodeled tasks, versus prior Opus versions' single-digit gains
- Fable's advantage comes from following a "golden workflow" (starting in the semantic layer, cross-checking raw SQL) and transparently stating assumptions, which lets it catch errors like a cents-for-dollars mistake that Opus misses
- On Hex's new "Frontier" benchmark for long-horizon, open-ended tasks, Fable at Max Effort hits 58% pass rate, notably outperforming other setups